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What is claimed is : 

1. A method of tracking moving objects in time-series 
pictures with processing the pictures, each picture being 
divided into blocks, each block consisting of a plurality of 
pixels, wherein in a case where an identification code of 
moving object is assigned in a unit of block, and a motion 
vector of moving object is determined in a unit of block, 
the method comprising the step of: 

(a) assigning the same identification code to adjacent 
blocks if an absolute value of a difference between motion 
vectors of the adjacent blocks is less than a predetermined 
value, thereby allowing different identification codes to be 
assigned to different moving objects overlapping in a 
picture . 

2. The method according to claim 1, wherein the 
processing of the step (a) is performed for each of N (N>2) 
consecutive pictures within the time-series pictures, the 
method further comprising the step of: 

(b) judging whether or not: a first object consisting 
of a group of blocks assigned a first identification code 
and a second object consisting of a group of blocks assigned 
a second identification code are in contact with each other 
in each of the N consecutive pictures; and a correlation 
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between the first objects in every temporally adjacent 
pictures in the N consecutive pictures is more than a 
predetermined value . 

3. The method according to claim 2, wherein the 
correlation is approximately equal to a ratio of an area of 
a logical AND between a figure obtained by moving one of the 
temporally adjacent first figures on the basis of a motion 
vector thereof and the other of the temporally adjacent 
first figures to an area of the one or the other of the 
temporally adjacent first figures. 

4. The method according to selective claim 2 or 3, 
further comprising the step of: 

(c) tracking the first and second objects backward in 
time after the judgment of the step (b) is positive. 

5. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 
wherein the program make the processor read and 
process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 
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divided into blocks, each block consisting of a plurality of 
pixels, an identification code of moving object is assigned 
in a unit of block, and a motion vector of moving object is 
determined in a unit of block, the program comprising the 
step of: 

(a) assigning the same identification code to adjacent 
blocks if an absolute value of a difference between motion 
vectors of the adjacent blocks is less than a predetermined 
value, thereby allowing different identification codes to be 
assigned to different moving objects overlapping in a 
picture . 

6. The apparatus according to claim 5, wherein the 
program further comprising the steps of: 

performing the processing of the step (a) for each of 
N (N>2) consecutive pictures within the time-series 
pictures; and 

(b) judging whether or not: a first object consisting 
of a group of blocks assigned a first identification code 
and a second object consisting of a group of blocks assigned 
a second identification code are in contact with each other 
in each of the N consecutive pictures; and a correlation 
between the first objects in every temporally adjacent 
pictures in the N consecutive pictures is more than a 
predetermined value. 



7. The apparatus according to claim 6, wherein the 
correlation of the step (b) is approximately equal to a 
ratio of an area of a logical AND between a figure obtained 
by moving one of the temporally adjacent first figures on 
the basis of a motion vector thereof and the other of the 
temporally adjacent first figures to an area of the one or 
the other of the temporally adjacent first figures. 

8. The apparatus according to any one of claim 6 or 7, 
the program further comprising the step of: 

(c) tracking the first and second objects backward in 
time after the judgment of the step (b) is positive. 

9. A method of tracking moving objects in time-series 
pictures with processing the pictures, each picture being 
divided into blocks, each block consisting of a plurality of 
pixels, wherein an identification code of moving object is 
assigned in a unit of block, and a motion vector of moving 
object is determined in a unit of block, the method 
comprising the steps of, when a first block whose motion 
vector is not determined is present, : 

(a) picking up determined motion vectors of blocks 
surrounding the first block; 

(b) classifying the determined motion vectors into 



groups such that an absolute value of a difference between 
any two motion vectors in the same group is less than a 
predetermined value; and 

(c) estimating a motion vector of the first block to 
be approximately equal to an average of motion vectors 
belonging to one, having the largest number of motion 
vectors, of the classified groups. 

10. The method according to claim 9, wherein when 
there is no determined motion vector to be picked up at the 
step (a) , the motion vectors estimated at the step (c) are 
regarded as determined motion vectors, and then the steps 
(a) to (c) are performed. 

11. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 
wherein the program make the processor read and 
process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 
divided into blocks, each block consisting of a plurality of 
pixels, an identification code of moving object is assigned 
in a unit of block, and a motion vector of moving object is 



determined in a unit of block, the program comprising the 
step of, when a first block whose motion vector is not 
determined is present, : 

(a) picking up determined motion vectors of blocks 
surrounding the first block; 

(b) classifying the determined motion vectors into 
groups such that an absolute value of a difference between 
any two motion vectors in the same group is less than a 
predetermined value; and 

(c) estimating a motion vector of the first block to 
be approximately equal to an average of motion vectors 
belonging to one, having the largest number of motion 
vectors, of the classified groups. 

12. The apparatus according to claim 11, wherein the 
program further comprises the step of: when there is no 
determined motion vector to be picked up at the step (a) , 
regarding the motion vectors estimated at the step (c) as 
determined motion vectors, and then performing the steps (a) 
to (c) . 

13. A method of tracking moving objects in time-series 
pictures with processing the pictures, each picture being 
divided into blocks, each block consisting of a plurality of 
pixels, wherein in a case where an identification code of 



moving object is assigned in a unit of block, and a motion 
vector of moving object is determined in a unit of block, 
the method comprising the steps of: 

(a) estimating a motion vector from a block-size 
region in a picture at a time tl to a region of interest in 
a picture at a time t2 as MV, and estimating an 
identification code of the block of interest as ID; 

(b) determining a correlation-related amount including 
an absolute value of a difference between the estimated 
motion vector MV of the block of interest and a motion 
vector of at least one, having an identification code equal 
to ID, of blocks surrounding the block of interest in the 
picture at the time t2; and 

(c) determining a value of an estimation function 
including the correlation-related amount for each first 
region moved within a predetermined range, and determining 
the motion vector MV and the identification code on the 
basis of an approximately-optimum value of the estimation 
function . 

14. The method according to claim 13, wherein the 
correlation-related amount of the step (c) is expressed by 
S[MV - MVneighbor I /L/ where MV ne ighbor denotes a motion vector of 
a block having the same identification code as the 
identification code ID of the block of interest, within 
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blocks surrounding the block of interest, £ denotes a sum 
over the blocks having said same identification code ID, and 
L denotes a number of the blocks having said same 
identification code ID . 

15. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 

wherein the program make the processor read and 
process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 
divided into blocks, each block consisting of a plurality of 
pixels, in a case where an identification code of moving 
object is assigned in a unit of block, and a motion vector 
of moving object is determined in a unit of block, the 
program comprising the step of: 

(a) estimating a motion vector from a block-size 
region in a picture at a time tl to a region of interest in 
a picture at a time t2 as MV, and estimating an 
identification code of the block of interest as ID; 

(b) determining a correlation-related amount including 
an absolute value of a difference between the estimated 
motion vector MV of the block of interest and a motion 
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vector of at least one, having an identification code equal 
to ID, of blocks surrounding the block of interest in the 
picture at the time t2; and 

(c) determining a value of an estimation function 
including the correlation-related amount for each first 
region moved within a predetermined range, and determining 
the motion vector MV and the identification code on the 
basis of an approximately-optimum value of the estimation 
function . 

16. The apparatus according to claim 15, wherein the 
correlation-related amount of the step (c) is expressed by 
S|MV - MV n eighbor I /L, where MV neighbor denotes a motion vector of 
a block having the same identification code as the 
identification code ID of the block of interest, within 
blocks surrounding the block of interest, S denotes a sum 
over the blocks having said same identification code ID, and 
L denotes a number of the blocks having said same 
identification code ID. 

17. A method of tracking moving objects in time-series 
pictures with processing the pictures, each picture being 
divided into blocks, each block consisting of a plurality of 
pixels, wherein in a case where an identification code of 
moving object is assigned in a unit of block, and a motion 
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vector of moving object is determined in a unit of block, 
the method comprising the steps of: 

estimating a motion vector from a block-size region in 
a picture at a time tl to a region of interest in a picture 
at a time t2 as MV, and determining a similarity-related 
amount between an image of a first region, which is 
concentric with the block-size region and is larger than the 
block-size region, and an image of a second region, which is 
concentric with the block of interest and is in the same 
form as the first region; and 

determining a value of an estimation function 
including the similarity-related amount for each first 
region moved within a predetermined range, and determining 
the motion vector MV on the basis of an approximately 
optimum value of the estimation function. 



18. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 

wherein the program make the processor read and 
process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 
divided into blocks, each block consisting of a plurality of 
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pixels, an identification code of moving object is assigned 
in a unit of block, and a motion vector of moving object is 
determined in a unit of block, the program comprising the 
step of: 

estimating a motion vector from a block-size region in 
a picture at a time tl to a region of interest in a picture 
at a time t2 as MV, and determining a similarity-related 
amount between an image of a first region, which is 
concentric with the block-size region and is larger than the 
block-size region, and an image of a second region, which is 
concentric with the block of interest and is in the same 
form as the first region; and 

determining a value of an estimation function 
including the similarity-related amount for each first 
region moved within a predetermined range, and determining 
the motion vector MV on the basis of an approximately 
optimum value of the estimation function. 

19. A method of tracking moving objects in time-series 
pictures with processing the pictures, the method comprising 
the steps of : 

(a) dividing each picture into blocks, each block 
consisting of a plurality of pixels; and 

(b) with regarding a background image as a moving 
object, assigning an identification code of moving object in 
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a unit of block and determining a motion vector of the 
moving object in a unit of block. 

20. The method according to claim 19, wherein the step 
(b) comprises the steps of: 

(bl) determining motion vectors of respective blocks 
in a picture at a time t2 through the use of block matching 
between a picture at a time tl and the picture at the time 
t2, without discriminating between the background image and 
moving objects; 

(b2) determining motion vectors of blocks, which are 
not determined at the step (bl), by estimating the motion 
vectors through the use of the method according to any one 
of claims 9, 13 or 14; and 

(b3) assigning the same identification code to 
adjacent blocks if an absolute value of a difference between 
motion vectors of the adjacent blocks is less than a 
predetermined value. 

21. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 
wherein the program make the processor read and 
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process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 
divided into blocks, each block consisting of a plurality of 
pixels, the program comprising the step of: 

(bl) determining motion vectors of respective blocks 
in a picture at a time t2 through the use of block matching 
between a picture at a time tl and the picture at the time 
t2, without discriminating between the background image and 
moving objects; 

(b2) determining motion vectors of blocks, which are 
not determined at the step (bl) , by estimating the motion 
vectors through the use of the method according to any one 
of claims 9, 13 or 14; and 

(b3) assigning the same identification code to 
adjacent blocks if an absolute value of a difference between 
motion vectors of the adjacent blocks is less than a 
predetermined value . 

22. A method of tracking a moving object in time- 
series pictures with processing the pictures, each picture 
being divided into blocks, each block consisting of a 
plurality of pixels, 

wherein a plurality of object maps of different times 
have been stored, each object map having motion vectors of 
the moving object in a unit of block, the method comprising 



the steps of: 

(a) determining a motion vector of a region of 
interest for one of the plurality of object maps; and 

(b) determining a motion vector of a region, to which 
the region of interest is moved with using the determined 
motion vector in positive or negative direction thereof, on 
the basis of an object map at a time corresponding to 
completion of the movement of the region, 

wherein the moved region is set as a region of 
interest on the object map of the time corresponding to the 
completion of the movement of the region, and the step (b) 
is repeated a plurality of times to track the region of 
interest. 

23. The method according to claim 22, wherein at the 
step (a) or the step (b) , a weighted motion vector average 
is determined as a motion vector of the region of interest 
with using motion vectors of blocks overlapping with the 
region of interest, where weights given to the respective 
motion vectors correspond to the number of pixels of 
respective portions overlapping between the respective 
blocks and the region of interest. 

24. The method according to any one of claim 22 or 23, 
wherein the region of interest of the step (a) corresponds 
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to one block. 

25. The method according to any one of claim 22 or 23, 
wherein the object map of the step (a) is a new object map, 
and at the step (b) , the region of interest is moved in the 
negative direction of the motion vector. 

26. The method according to any one of claim 22 or 23, 
wherein the number of the plurality of object maps is kept 
constant by updating an oldest object map with a newest 
object map. 

27. The method according to any one of claim 22 or 23, 
wherein, on the basis of both a region of interest at a time 
tl and a corresponding region of interest at a time t2, a 
motion vector from the time tl to the time t2 is determined 
as a fast-forward motion vector, and 

wherein if a difference of an absolute value between 
respective fast-forward motion vectors of two adjacent 
regions of interest on the object map of the time t2 is more 
than a predetermined value, the two adjacent regions of 
interest are recognized as different moving objects. 

28. The method according to claim 27, wherein, if a 
plurality of peaks are present in a histogram of absolute 
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values of motion vectors for a single cluster including 
adjacent blocks having motion vectors, an interval between 
the times tl and t2 is determined on the basis of a speed 
difference between the peaks. 

29. The method according to claim 27, wherein each 
time the interval between the times tl and t2 is increased, 
it is determined whether or not the absolute value of the 
difference between the respective fast-forward motion 
vectors of the two adjacent regions of interest on the 
object map of the time t2 is more than the predetermined 
value, and 

wherein if the interval is less than a predetermined 
maximum value and if the absolute difference is more than 
the predetermined value, then the two adjacent regions of 
interest are recognized as different moving objects. 

30. An apparatus for tracking moving objects in time- 
series pictures, comprising: 

a storage device for storing the time-series pictures 
and a program; and 

a processor coupled to the storage device, 
wherein the program make the processor read and 
process the time-series pictures to track the moving objects 
in the pictures, and by the processing, each picture is 



divided into blocks, each block consisting of a plurality of 
pixels, a plurality of object maps of different times have 
been stored, each object map having motion vectors of the 
moving object in a unit of block, the program comprising the 
step of: 

(a) determining a motion vector of a region of 
interest for one of the plurality of object maps; and 

(b) determining a motion vector of a region, to which 
the region of interest is moved with using the determined 
motion vector in positive or negative direction thereof, on 
the basis of an object map at a time corresponding to 
completion of the movement of the region, 

wherein the moved region is set as a region of 
interest on the object map of the time corresponding to the 
completion of the movement of the region, and the step (b) 
is repeated a plurality of times to track the region of 
interest . 

31. The apparatus according to claim 30, wherein at 
the step (a) or the step (b) , a weighted motion vector 
average is determined as a motion vector of the region of 
interest with using motion vectors of blocks overlapping 
with the region of interest, where weights given to the 
respective motion vectors correspond to the number of pixels 
of respective portions overlapping between the respective 
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blocks and the region of interest. 

32. The apparatus according to any one of claim 30 or 
31, wherein the region of interest of the step (a) 
corresponds to one block. 

33. The apparatus according to any one of claim 30 or 
31, wherein the object map of the step (a) is a new object 
map, and at the step (b) , the region of interest is moved in 
the negative direction of the motion vector. 

34. The apparatus according to any one of claim 30 or 
31, wherein the program further comprises the step of: 

keeping the number of the plurality of object maps 
constant by updating an oldest object map with a newest 
object map. 

35. The apparatus according to any one of claim 30 or 
31, wherein the program further comprises the step of: 

on the basis of both a region of interest at a time tl 
and a corresponding region of interest at a time t2, 
determining a motion vector from the time tl to the time t2 
as a fast-forward motion vector; and 

if a difference of an absolute value between 
respective fast-forward motion vectors of two adjacent 
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regions of interest on the object map of the time t2 is more 
than a predetermined value, recognizing the two adjacent 
regions of interest as different moving objects. 

36. The apparatus according to claim 35, wherein the 
program further comprises the step of: 

if a plurality of peaks are present in a histogram of 
absolute values of motion vectors for a single cluster 
including adjacent blocks having motion vectors, determining 
an interval between the times tl and t2 on the basis of a 
speed difference between the peaks. 

37. The apparatus according to claim 35, wherein the 
program further comprises the step of: 

each time the interval between the times tl and t2 is 
increased, determining whether or not the absolute value of 
the difference between the respective fast-forward motion 
vectors of the two adjacent regions of interest on the 
object map of the time t2 is more than the predetermined 
value; and 

if the interval is less than a predetermined maximum 
value and if the absolute difference is more than the 
predetermined value, recognizing the two adjacent regions of 
interest as different moving objects. 



